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DETAILED ACTION 

Priority 

1. Receipt is acknowledged of papers submitted under 35 U.S. C. 119(a)-(d), which 
papers have been placed of record in the file. 



Drawings 

2. The drawings are objected to because of following minor informalities: 
The drawings 3, 4, and 7 are objected to as failing to comply with 37 

CFR 1 .84(p)(5) because they do include the following reference character "X" not 

mentioned in the description. 

The drawing 10 (a) is objected for the label 55b has no corresponding data block 

identified. 

Corrected drawing sheets in compliance with 37 CFR 1 .121(d) are required in 
reply to the Office action to avoid abandonment of the application. Any amended 
replacement drawing sheet should include all of the figures appearing on the immediate 
prior version of the sheet, even if only one figure is being amended. The figure or figure 
number of an amended drawing should not be labeled as "amended." If a drawing figure 
is to be canceled, the appropriate figure must be removed from the replacement sheet, 
and where necessary, the remaining figures must be renumbered and appropriate 
changes made to the brief description of the several views of the drawings for 
consistency. Additional replacement sheets may be necessary to show the renumbering 
of the remaining figures. Each drawing sheet submitted after the filing date of an 
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application must be labeled in the top margin as either "Replacement Sheet" or "New 
Sheet" pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, 
the applicant will be notified and informed of any required corrective action in the next 
Office action. The objection to the drawings will not be held in abeyance. 

Specification 

3. The title of the invention, "Method of generating a labeled image and image 
processing system" is too general to reveal the real intention to which the claims are 
directed. A new title is suggested: "Method of generating a labeled image and image 
processing system with pixel blocks". 

4. The disclosure is objected to because of the following informalities and 
appropriate corrections are required: 

a) Page 12, line 21 : "other pixels that have already been labeled with provisional 
identification information". It is unclear how and why only certain pixels not all 
binarized pixels given, i.e. already been labeled, provisional identification 
information (PID). 

b) Page 12, lines 23-28. Examiner believes that important invention concepts are 
imbedded in these lines. Unfortunately the wording is vague and confusing such 
as: "identifier of PID is connected to other identifier ... at the later stage". What 
is the meaning of the identifier? How and at what stage of the binarized image 
processing have these identifiers been assigned? Under what conditions 
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identifiers connected to each other? Furthermore, at page 12, line 27, the term 
"inherit" appears after identifier without explanation of the meaning of inherit. It is 
recommended to have a flow diagram to show the relationships among: binarized 
pixel value, provisional identification information, common identification 
information, identifier, labeling, connection, inherit, the generation of connecting 
information, and relabeling, 
c) Page 14, lines 12-17, why there are two distinct requirements for the same set of 
figures 3(a) to 3(d), wherein one requirement is based on both the states of 
group 4 and the states of block 2 and another requirement is based solely on the 
state of gO of block 2? Also, what is the relationship between on/off-pixel and 
provisional identifier? Also, according to: "a new provisional identifier will be 
assigned to the pixel g3" at page 14, line 22. Is this means that for that pixel 
block only the pixel g3 got identifier? If this is true then it will contradict to the 
concept of parallel which states "the pixels are labeled in parallel with the same 
identifier" at page 14, line 3. Please clarify the concept of labeled in parallel for 
the pixel block. 

Claim Rejections - 35 USC § 101 
5. 35 U.S.C. 101 reads as follows: 



Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 
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The USPTO "Interim Guidelines for Examination of Patent Applications for Patent 
Subject Matter Eligibility" (Official Gazette notice of 22 November 2005), Annex IV, 
reads as follows (see also MPEP 2106): 

Descriptive material can be characterized as either "functional descriptive material" or 
"nonfunctional descriptive material." In this context, "functional descriptive material" 
consists of data structures and computer programs which impart functionality when 
employed as a computer component. (The definition of "data structure" is "a physical 
or logical relationship among data elements, designed to support specific data 
manipulation functions." The New IEEE Standard Dictionary of Electrical and 
Electronics Terms 308 (5th ed. 1993).) "Nonfunctional descriptive material" includes 
but is not limited to music, literary works and a compilation or mere arrangement of 
data. 

When functional descriptive material is recorded on some computer-readable 
medium it becomes structurally and functionally interrelated to the medium and will 
be statutory in most cases since use of technology permits the function of the 
descriptive material to be realized. Compare In re Lowry, 32 F.3d 1579, 1583-84, 32 
USPQ2d 1031, 1035 (Fed. Cir. 1994) (claim to data structure stored on a computer 
readable medium that increases computer efficiency held statutory) and Warmerdam, 
33 F.3d at 1 360-61 , 31 USPQ2d at 1 759 (claim to computer having a specific data 
structure stored in memory held statutory product-by-process claim) with 
Warmerdam, 33 F.3d at 1 361 , 31 USPQ2d at 1 760 (claim to a data structure per se 
held nonstatutory). 

In contrast, a claimed computer-readable medium encoded with a computer program 
is a computer element which defines structural and functional interrelationships 
between the computer program and the rest of the computer which permit the 
computer program's functionality to be realized, and is thus statutory. See Lowry, 32 
F.3d at 1583-84, 32 USPQ2d at 1035. 



Claim 17 (and therefore claims 18-30 by dependency) is rejected under 35 
U.S.C. 101 because the claimed invention is directed to non-statutory subject matter as 
follows. Claim 17 define as a "system". However, while the preamble defines a 
"system", which would typically be indicative of an "apparatus", the body of the claim 
lacks definite structure indicative of a physical apparatus. Furthermore, the 
specification indicates that the invention may be embodied as software (specification 
page 22, lines 7-8). Therefore, the claim as a whole appears to be nothing more than a 
"system" of software elements, thus defining functional descriptive material perse. 
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Functional descriptive material may be statutory if it resides on a non-transitory 
"computer-readable medium or computer-readable memory". The claim(s) indicated 
above lack structure, and do not define a non-transitory computer readable medium and 
are thus non-statutory for that reason (i.e., "When functional descriptive material is 
recorded on some computer-readable medium it becomes structurally and functionally 
interrelated to the medium and will be statutory in most cases since use of technology 
permits the function of the descriptive material to be realized" - Guidelines Annex IV). 
The scope of the presently claimed invention encompasses products that are not 
necessarily computer readable, and thus NOT able to impart any functionality of the 
recited program. The examiner suggests: 

1 . Amending the claim(s) to embody the program on "non-transitory 
computer-readable medium" or equivalent; or 

2. Adding structure to the body of the claim that would clearly define a 
statutory apparatus. 

Any amendment to the claim should be commensurate with its 
corresponding disclosure. 
Note: 

"A transitory, propagating signal ... is not a "process, machine, manufacture, or 
composition of matter." Those four categories define the explicit scope and reach of 
subject matter patentable under 35 U.S.C. § 101 ; thus, such a signal cannot be 
patentable subject matter." (In re Nuijten, 84 USPQ2d 1495 (Fed. Cir. 2007)). 
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Should the full scope of the claim as properly read in light of the disclosure 
encompass non-statutory subject matter such as a "signal", the claim as a whole would 
be non-statutory. Should the applicant's specification define or exemplify the computer 
readable medium or memory (or whatever language applicant chooses to recite a 
computer readable medium equivalent) as statutory tangible products such as a hard 
drive, ROM, RAM, etc, as well as a non-statutory entity such as a "signal", "carrier 
wave", or "transmission medium", the examiner suggests amending the claim to include 
the disclosed tangible non-transitory computer readable storage media, while at the 
same time excluding the intangible transitory media such as signals, carrier waves, etc. 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claims 1-3 and 10-16 are rejected under 35 U.S.C. 103(a) as being obvious by 
Murakawa et al. (US 2003/0156757). 

Regarding claim 1, Murakawa discloses an image processing system comprising: 
- inputting a pixel block, which includes a plurality of pixels that are adjacent to 
one another in more than one dimension, as a single unit from data including pixels for 
forming an image ("In a first aspect of the invention, an image processing apparatus is 
provided for retrieving a specific pattern of an image. The apparatus comprises a unit 
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for dividing the image into a plurality of block regions, each of the block regions having a 
predetermined size" in paragraph 16, line 1. "The input image 20 (figure 4) is divided 
into some block regions each having a predetermined size. The block size can be set 
to an optional size (mxn pixels)" in paragraph 59, line 5. As shown in figure 4 one pixel 
block includes a plurality of pixels that are adjacent to one another in two dimensions); 

- labeling, based on binarized pixels, all on-pixels or all off-pixels that are 
subjects for grouping and are included in the pixel block with common identification 
information ("The feature is obtained for each pixel in the block region, and the pixel is 
labeled when the amount of feature is within a predetermined range. Then, the number 
of the labeled pixels in the block region is counted as the feature amount in the block. 
The feature includes a) the number of pixels within a specific range in a color space, b) 
the strength of an edge, c) the co-occurrence characteristic obtained by conversion to a 
binary image and the like" in paragraph 60, line 5. See also, 'the image in the block 
region is converted into a binary image with a certain threshold, and then the co- 
occurrence characteristic of each pixel with the adjacent pixel is set to the feature 
amount. The co-occurrence characteristic represents a degree of gathering of the 
pixels having the same pixel value and indicates a probability that a certain pixel has the 
same pixel value as the value of a pixel adjacent thereto" in paragraph 66, line 3. Thus, 
the co-occurrence characteristic is the common identification information which is the 
base for labeling. "Moreover, the connecting region is detected one by one for the 
labeling. Therefore in the environment capable of executing the parallel processing, in 
the image labeling process" in paragraph 108, line 11. 
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Murakawa discloses the usage of a binary image. Murakawa does not explicitly 
disclose the on-pixels and off-pixels. It would have been obvious for a person of 
ordinary skill in the art of image processing to recognize that the binarization of an 
image is the process to convert pixel to either 1 , the on-pixel or 0, the off-pixel). 

Regarding claim 2, the pixel block is composed of four pixels adjacent to one 
another in two dimensions (as discussed in claim 1 by Murakawa that an image 
processing system can have optional pixel block with size of mxn pixels. Thus, 
Murakawa teaches that a pixel block can have four pixels adjacent to one another in two 
dimensions). 

Regarding claim 3: - a first stage (as depicted in Murakawa figure 5 with S301 for 
inputting data) of scanning the image (figure 1, numeral 8), labeling with provisional 
identification information, and generating connecting information for the provisional 
identification information (as discussed in claim 1 the labeling is based on the common 
identification information which is the provisional identification information as a degree 
of gathering of pixels, i.e. the connecting information); 

- a second stage (as depicted in Murakawa figure 16, labeling pixels in 
connecting region with S1301 for inputting data) of labeling with real identification 
information showing image elements based on the connecting information (as 
discussed by Murakawa paragraphs 93-97, "... The above mentioned processes are 
repeated until the whole image 21 is examined and all the connecting regions included 
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in the image are detected to be labeled (S1307 to S1314)" in paragraph 97, line 22. 
Thus, the image to be labeled is based on connecting information as the real 
identification information); 

- the first stage and the second stage each include the step of inputting and the 
step of labeling (as discussed above for the first stage and the second stage each 
includes inputting and labeling); 

- in the step of labeling of the first stage, the provisional identification information 
is the common identification information for labeling (as discussed in the first stage, the 
provisional identification information is the common identification information for 
labeling); 

- in the step of labeling of the second stage, the provisional identification 
information is the common identification information for labeling (as discussed in the 
second stage, the real identification information is the common identification information 
for labeling). 

Regarding claim 10: - inputting a pixel block, which includes a limited number of 
pixels that are adjacent to one another in more than one dimension, as a single unit 
from data including a plurality of pixels for forming an image (discussed in claim 1 for 
inputting a pixel block); 

- labeling, based on binarized pixels, all on-pixels or all off-pixels that are subject 
for grouping included in a pixel block with common identification information (discussed 
in claim 1 for labeling); 
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- calculating characteristic values of respective image elements (the feature 
characteristic calculation includes: "... a) the number of pixels within a specific range in 
a color space, b) the strength of an edge, c) the co-occurrence characteristic obtained 
by conversion to a binary image and the like" in Murakawa paragraph 60, line 9. See 
also, "When the specific pattern has a uniform co-occurrence characteristic within a 
certain error range for the block size, the image in the block region is converted into a 
binary image with a certain threshold" in Murakawa paragraph 66, line 1, wherein the 
specific pattern is a group of connected pixels is the image element) repeatedly carrying 
out an operation in units that include at least one pixel block (as depicted in Murakawa 
figure 5, the repeating processes S303 to S308). 

Regarding claim 11:- including a step of calculating a block characteristic value 
that contributes to a characteristic value of an image element (discussed in claim 10 for 
calculating characteristic values); 

- the step of calculating a block characteristic value being performed in parallel 
with the step of labeling and in units of the pixel blocks under labeling ("the connecting 
region is detected one by one for the labeling. Therefore in the environment capable of 
executing the parallel processing, in the image labeling process, for example, a 
predetermined process including image recognition process can be carried out for the 
detected one connecting region, while the other connecting regions can be detected" in 
paragraph 108, line 11). 
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Regarding claim 12, the step of labeling includes scanning the image for labeling 
with provisional identification information (discussed in claim 3 for a first stage of 
scanning). 

Regarding claim 13, the calculating a block characteristic value includes 
calculating the block characteristic value based on multivalue pixels included in the pixel 
block under labeling (discussed in claim 10 for the block characteristic value calculation 
includes pixel value in color space which is a multivalue space). 

Regarding claim 14, in the inputting, the pixel block composed of four pixels 
adjacent to one another in two dimensions (discussed in claim 2). 

Regarding claim 15, in the step of inputting, in addition to the pixel block 
composed of four pixels adjacent to one another in two dimensions (discussed in claim 
2), a large pixel block composed of four pixel blocks adjacent to one another in two 
dimensions is inputted as another single unit (as discussed in claim 1 by Murakawa for 
block size can be set to an optional size (mxn pixels). Thus, the optional size can be a 
large pixel block composed of four pixel blocks adjacent to one another in two 
dimensions is inputted as another single unit), and in the step of labeling, all on-pixels or 
all off-pixels that are subject for grouping included in the large pixel block are labeled 
with the common identification information (discussed in claims 1 and 10 for labeling). 
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Regarding claim 16: - inputting a pixel block, which includes a plurality of pixels 
that are adjacent to one another in more than one dimension, as a single unit from data 
including pixels for forming an image (discussed in claim 1 for inputting a pixel block); 

- labeling, based on binarized pixels, all on-pixels or all off-pixels that are subject 
for grouping and are included in the pixel block with same identification information 
(discussed in claim 1 for labeling); 

- distinguishing image elements in a labeled image ("an image processing 
method is provided for retrieving a specific pattern of an image. The method comprises 
dividing the image into a plurality of block regions ... detecting a predetermined pattern 
for specifying a search region on the image from the labeled map image, specifying the 
search region based on a position on the map image in which the predetermined pattern 
is detected, and retrieving the specific pattern in the specified search region on the 
image" in paragraph 18, line 1 , wherein the specific pattern is an image element of an 
image which retrieved from labeled image). 

8. Claims 17-21 and 28-30 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the prior art Murakawa et al. (US 2003/01 56757) as applied to claim 
1 discussed above, and further in view of Sakagami et al. (US 2003/01 1 0444). 

Regarding claim 17, Murakawa discloses an image processing system 
comprising: 

- an interface configured for inputting data including a plurality of pixels, which 
are adjacent in more than one dimension and constitute a pixel block, including pixels 
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for forming an image (as discussed in claim 1 by Murakawa for inputting a pixel block. 
As depicted in Murakawa figure 2, numeral 8, scanner as scan in data and numeral 201 , 
CPU, to form an image. Thus, #8 and #201 form the data inputting interface to receive 
and form image data); 

- a labeling processor configured for labeling, based on binarized pixels, all on- 
pixels or all off-pixels that are subject for grouping and are included in the pixel block 
with common identification information in parallel (as discussed in claim 1 by Murakawa 
for labeling. As depicted in Murakawa figure 2, numeral 201 , the CPU is the labeling 
processor). 

Murakawa does not explicitly disclose the data inputting is in parallel. 

Sakagami, in the field of endeavor of data processing ("recording a multi-level 
signal on and reproducing a multi-level signal from an information recording medium" in 
paragraph 2, line 4), teaches the concept of parallel inputting as shown in figure 22: 
"The data processing circuit includes first through (n-k)the A-selectors 100-1 through 
100-(n-k)" in Sakagami paragraph 344, line 1. See also, "Thus, the (nxm-l)-bit parallel 
data is input to the data processing circuit of the 12 th embodiment, and the data 
processing circuit outputs parallel data of n bits per symbol. "Further, the data 
processing circuit of the 12 th embodiment may include a shift register circuit so that 
serial data as well as parallel data may be input to the data processing circuit" in 
paragraph 352, line 1. 

It would have been obvious at the time the invention was made, that one of 
ordinary skill in the art would have been motivated to modify and provide the said image 
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processing system Murakawa made, with the application of parallel data inputting as 
taught by Sakagami, such that a system can receive data not only in serial mode but 
also in parallel mode as discussed above by Sakagami. 

Regarding claim 18, the pixel block is composed of four pixels adjacent to one 
another in two dimensions (discussed in claim 2). 

Regarding claim 19: - a processor including a processing region (Murakawa 
figure 2, numeral 201 ) that includes a plurality of processing elements, a plurality of data 
paths that operate in parallel being configured by the plurality of processing elements in 
the processing region (as discussed in claim 17 by Sakagami for the parallel operation. 
As depicted in Sakagami figure 22, plurality of processing elements 100-1 through 100- 
(n-k) with plurality of data paths); 

- the interface and the labeling processor are configured in the processing region 
(discussed in claim 17). 

Regarding claim 20: - a first processing system for scanning the image, labeling 
with provisional identification information, and generating connecting information for the 
provisional identification information (discussed in claim 3 for a first stage); 

- a second processing system for labeling with real identification information 
showing image elements based on the connecting information (discussed in claim 3 for 
a second stage); 
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- the first processing system and the second processing system respectively 
include the interface and the labeling processor (discussed in claim 3, the first 
processing system and the second processing system each includes interface and 
labeling. See also the discussion in claim 17 for the interface and labeling processor); 

- the labeling processor of the first processing system assigns the provisional 
identification information as the common identification information for labeling 
(discussed in claim 3 for the labeling of the first stage); 

- the labeling processor of the second processing system assigns the real 
identification information as the common identification information for labeling 
(discussed in claim 3 for the labeling of the second stage). 

Regarding claim 21 : - a reconfigurable processor including a processing region 
that includes a plurality of processing elements, a plurality of data paths that operate in 
parallel being configured by the plurality of the processing elements in the processing 
region, and a control unit for reconfiguring the processing region (discussed in claim 19 
wherein Murakawa figure 2, numeral 201 is the reconfigurable processor and the control 
unit); 

- the interface and the labeling processor included in the first processing system 
and the interface and the labeling processor included in the second processing system 
are configured at different timing in the processing region (as discussed in claim 20 that 
the first processing system generating connecting information and the second 
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processing system using the generated connection information, thus, these two systems 
are configured at different timing). 

Regarding claim 28, first processor configured to repeatedly performing an 
operations in units of at least one pixel block to calculate a characteristic value of each 
image element (discussed in claim 10 for calculating characteristic values). 

Regarding claim 29: a second processor configured to, data including a pixel 
block being supplied to the second processor by the interface in parallel with the 
labeling processor, calculate a block characteristic value that contributes to a 
characteristic value of an image element in units of the pixel blocks under labeling (the 
feature characteristic calculation includes: "... a) the number of pixels within a specific 
range in a color space, b) the strength of an edge, c) the co-occurrence characteristic 
obtained by conversion to a binary image and the like" in Murakawa paragraph 60, line 
9. See the discussion in claims 1 and 17 for labeling. See also the parallel processing 
in claim 17). 

Regarding claim 30, the second processor is configured to calculate values that 
contribute to the characteristic values of image elements from multivalue pixels included 
in pixel blocks under labeling (discussed in claim 13). 
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Examiner's Comments 

9. The following is examiner's comments about claims 4-9 and 22-27: 

10. Claim 4 is objected to as being dependent upon a rejected base claim, but would 
be allowable if rewritten in independent form including all of the limitations of the base 
claim and any intervening claims. 

1 1 . Claim 22 is rejected under USC §101 and would be considered to be allowable if 
amended to overcome the rejections set forth in this Office action above and rewritten in 
independent form including all of the limitations of the base claim and any intervening 
claims. 



Conclusion 



12. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Eueng-nan Yeh whose telephone number is 571-270- 
1586. The examiner can normally be reached on Monday-Friday 8AM-4 :30PM EDT. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Vikkram Bali can be reached on 571-272-7415. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 



Application/Control Number: 10/590,778 Page 19 

Art Unit: 2624 

Status information for unpublished applications is available through Private PAIR only. 

For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 

you have questions on access to the Private PAIR system, contact the Electronic 

Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 

USPTO Customer Service Representative or access to the automated information 

system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Eueng-nan Yeh 
Assistant Patent Examiner 
Art Unit: 2624 
IE X.I 

A/IKKRAM BALI/ 

Supervisory Patent Examiner, Art Unit 2624 



